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FOREWORD 


It  has  long  been  assumed  that  Individuals  will  not  respond 
candidly  to  sensitive  survey  questions  if  they  believe  "the  wrong 
people1*  will  find  out  how  they  responded  to  these  questions  and 
link  their  responses  to  them  personally.  Most  Army  surveys  as¬ 
sure  confidentiality  or  anonymity  on  the  assumption  that  includ¬ 
ing  such  assurances  frees  respondents  to  answer  the  questions 
candidly.  What  has  not  been  known,  however,  is  the  extent  to 
which  these  assurances  are  believed  by  the  soldiers  who  partici¬ 
pate  in  the  surveys. 

This  research  presents  evidence  that  soldiers  frequently  do 
not  believe  the  assurances  of  confidentiality  or  anonymity  they 
are  given  in  Army  surveys  and  indicates  some  of  their  reasons  for 
not  believing  them. 


EDGAR  M.  JOHNSON 
Director 
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AN  INVESTIGATION  OF  RESPONDENT  CONFIDENTIALITY  CONCERNS 
IN  ARMY  SURVEYS:  DATA  FROM  TEN  FOCUS  GROUPS 


EXECUTIVE  SUMMARY 


Requirement : 

It  has  become  standard  practice  for  organizations  conduct¬ 
ing  sample  surveys  to  provide  appropriate  guarantees  of  confiden¬ 
tiality  and,  in  some  cases,  anonymity.  This  research  sought 
information  on  whether  soldiers  believe  such  guarantees  and,  if 
not,  what  reasons  they  give  for  not  believing  them. 


Procedure : 

Participants  (£>*40) ,  meeting  in  small  homogeneous  groups, 
completed  a  standard  Army  survey  that  asked  questions  on  six 
topics  of  varying  sensitivity.  These  same  participants  also 
completed  a  3-item  questionnaire  asking  about  any  confidentiality 
concerns  they  may  have  had  while  filling  out  the  survey.  Then, 
with  their  completed  survey  forms  in  front  of  them,  they  took 
part  in  a  focus  group  discussion  to  elaborate  on  these  concerns. 


Findings : 

with  respect  to  the  first  part  of  the  objective,  responses 
to  an  objective  questionnaire  indicated  that  only  18  of  the  40 
participants  (45%)  routinely  believed  the  confidentiality  assur¬ 
ances  they  were  given  in  Army  surveys,  and  only  23  of  them  (57%) 
said  they  believed  the  assurances  they  were  given  in  the  present 
survey.  (This  57%  figure  is  similar  to  the  58%  figure  obtained 
in  a  recent  Armywide  survey.) 

With  respect  to  the  second  part  of  the  objective,  there 
appeared  to  be  three  interrelated  reasons  some  soldiers  have 
doubts  or  uncertainties  about  the  assurances  of  confidentiality 
they  are  given.  First,  they  believed  (quite  apart  from  the 
sincerity  they  were  willing  to  attribute  to  those  conducting  the 
survey)  that  it  is  simply  not  possible  to  guarantee  confidential¬ 
ity  for  survey  responses.  They  believed  that  survey  booklets  (or 
data  recorded  from  these  booklets)  always  stand  a  chance  of  being 
seen,  inadvertently  or  otherwise,  by  individuals  whom  the  respon¬ 
dents  would  not  want  to  see  them.  Second,  the  soldiers  believed 
that  the  nature  of  the  background  information  requested  in  many 
Army  surveys  is  such  that  the  various  pieces  of  information  could 
be  combined  (in  the  event  someone  wanted  to  do  so)  to  identify 

vii 


particular  individuals  and  connect  them  with  the  responses  they 
gave  to  the  substantive  items.  In  other  words,  in  the  minds  of 
some  soldiers  the  fact  that  they  are  not  personally  identified  in 
the  survey  booklet  does  not  mean  that  they  are  not  identifiable. 
And  third,  the  soldiers  had  difficulty  articulating  a  reason  for 
the  Army's  including  many  of  these  background  items;  in  this 
situation  many  assumed  the  worst.  Many  of  them  assumed  such 
items  to  have  been  included  because  someone  (it  wasn't  necessary 
to  be  able  to  say  whom)  wanted  to  be  able  to  identify  them 
individually. 


Utilization  of  Findings: 

A  portion  of  the  results  of  this  research  will  be  used  by 
the  Army  Personnel  Survey  Office  (APSO)  to  aid  decision  making 
about  relevant  matters  in  future  surveys.  The  results  will  also 
be  used  to  design  new  research  to  answer  some  of  the  questions 
raised. 
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AN  INVESTIGATION  OF  RESPONDENT  CONFIDENTIALITY  CONCERNS  IN  ARMY 
SURVEYS:  DATA  FROM  TEN  FOCUS  GROUPS 


INTRODUCTION 


The  Problem 

It  has  long  been  taken  as  a  given  that  individuals  will  not 
respond  candidly  to  sensitive  survey  questions  if  they  believe 
"the  wrong  people"  will  be  able  to  find  out  how  they  responded  to 
these  questions  and  link  their  responses  to  them  personally. 

For  some,  "the  wrong  people"  are  simply  any  individuals  and 
groups  other  than  those  sponsoring  and/or  conducting  the  survey, 
and  what  these  respondents  want  to  be  convinced  of  is 
confidentiality.  When  survey  instructions  tell  respondents  that 
their  responses  will  be  kept  confidential,  that  records  and 
reports  will  not  link  them  personally  to  the  data  they  provide, 
it  is  mainly  to  such  individuals  as  these  that  the  assurance  is 
being  directed.  For  others  "the  wrong  people"  are  almost  anyone 
with  the  ability  to  link  them  individually  to  their  responses 
(and  this  includes  those  professionally  associated  with  the 
survey) .  In  these  cases,  what  respondents  want  to  be  convinced 
of  is  anonymity.  And  when  survey  instructions  tell  respondents 
that  the  survey  is  anonymous  and  ask  them  not  to  put  their  name 
or  social  security  number  (SSN)  anywhere  on  the  form,  it  is 
mainly  to  such  individuals  as  these  that  the  assurance  is  being 
directed. 

It  has  become  a  general  practice  (in  the  case  of  Army  surveys, 
a  requirement)  for  organizations  conducting  sample  surveys  to 
provide  appropriate  guarantees  of  confidentiality  and  in  some 
cases  anonymity.1.  What  is  not  clear  is  the  extent  to  which 
these  guarantees  of  confidentiality  or  anonymity  are  believed  by 
the  individuals  who  participate  in  these  surveys. 

Over  the  years  a  number  of  studies  have  investigated  the 
effect  of  anonymity  and  confidentiality  assurances  on 
questionnaire  responses  (e.g.,  Ash  &  Abramson,  1952;  Boruch  & 
Cecil,  1979;  Corey,  1937;  Downs  &  Kerr,  1986;  Drake,  1989; 
Esposito,  Agard,  &  Rosnow,  1984;  Fischer,  1946;  Fuller,  1974; 
Futtrell  &  Swan,  1977;  Frey,  1986;  Fusilier,  &  Hoyer,  1980; 

Hamel  &  Reif,  1952;  Hartnett  &  Seligsohn,  1967);  Houston  & 
Jefferson,  1975;  Jones,  1979;  Manniche,  &  Hayes,  1957;  McGuire, 
1969;  Perlin,  1961;  Rosen,  1960;  Rosenberg,  1969;  Singer, 

Hippier,  &  Schwarz,  1993;  Singer  &  Miller,  1993;  Tyson,  & 
Kaplowitz,  1977;  Wildman,  1977;  Wilson  &  Rosen,  1975;  Woods  & 


xThere  are  some,  of  course,  who  are  unwilling  to  provide  any 
information  about  themselves  that  they  are  not  absolutely  required 
to  provide  (Singer  &  Miller,  1993) ;  but  such  individuals  are 
unlikely  to  respond  to  any  non -required  survey  items- -particularly 
if  they  are  asked  to  identify  themselves. 


McNamara,  1980).  As  Singer  and  her  colleagues  (e.g..  Singer, 
Hippier,  &  Schwarz,  1993)  have  pointed  out,  however,  these 
studies  have  not  shown  clearly  that  confidentiality  assurances 
improve  data  quality.  In  some  studies  there  has  been  no  effect 
(e.g.,  Hartnett  &  Seligsohn,  1967;  Reamer,  1987),  and  in  other 
studies  (e.g.,  experiments  in  which  the  topic  has  intentionally 
been  made  non-sensitive--see  Singer  et  al . ,  1993),  assurances  of 
confidentiality  have  produced  negative  effects  on  subjects' 
willingness  to  take  part  in  the  survey. 

Nearly  all  these  studies  were  conducted  by  organizations  that, 
unlike  the  Army,  have  little  or  no  control  over  those  who  respond 
to  their  survey;  and  virtually  all  of  the  studies  have  used  non¬ 
military  samples.2  The  implications  of  these  studies  for  Army 
research  on  this  topic  would  thus  appear  to  be  limited  to  two: 

(1)  Confidentiality  assurances  can't  always  be  counted  on  to 
reduce  respondents'  concerns  about  confidentiality  and  anonymity, 
and  (2)  in  some  situations  such  assurances  can  backfire. 


Objective 

This  research  addressed  the  first  of  these  two  implications. 
Its  objective  was  to  obtain  information  on  whether  soldiers 
believe  the  assurance  of  confidentiality  they  receive  in  a 
standard  Army  survey  and,  if  not,  what  reasons  they  give  for  not 
believing  it.  The  research  was  designed  less  to  test  ideas  than 
to  generate  them. 


METHOD 


Overview 


Participants  completed  a  standard  Army  survey  that  asked 
questions  on  six  topics  of  varying  sensitivity.  Afterwards  these 
same  participants  completed  a  3-item  questionnaire  asking  about 
any  confidentiality  concerns  they  may  have  had  while  filling  out 
the  survey.  Then,  with  their  completed  survey  forms  in  front  of 
them,  they  took  part  in  a  focus  group  discussion  aimed  at 
elaborating  these  concerns. 


2Wilson  and  Rosen  (1975)  collected  data  from  soldiers  on  the 
effects  of  response  anonymity.  The  experiment  was  what  might  be 
called  an  "imagining"  experiment.  Subjects  were  not  exposed  to 
actual  experimental  treatments;  rather,  they  were  told  about  each 
of  several  experiment  conditions  and  asked  to  say  how  they  would 
respond  under  each  of  these  conditions. 
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Participants 


Participants  were  40  soldiers:  16  officers  and  24  enlisted 
personnel,  as  shown  below: 

6  captains  (03) 

10  lieutenants  (01-02) 

3  senior  NCOS  (E8) 

10  junior  NCOS  (E6) 

11  junior  enlisted  (E2-E4) 

Thirty  were  male,  and  ten  were  female. 


Procedure 

In  each  of  10  small  groups,  homogeneous  in  rank/grade,  the 
Investigator  administered  an  omnibus-type  survey  including  items 
from  the  fall-1992  and  the  spring-1993  Sample  Survey  of  Military 
Personnel  (SSMP) .  The  instrument  took,  on  the  average,  about  28 
minutes  to  complete.  When  participants  had  completed  the 
instrument,  they  didn't  turn  it  in  immediately  but  retained  it  to 
refer  to  during  the  discussion  that  was  to  take  place  shortly. 
Participants  were  then  given  a  brief,  3 -item  questionnaire 
concerning  confidentiality  in  surveys  (Table  1) .  When  these  3- 
item  questionnaires  had  been  completed,  they  were  collected;  and 
the  Investigator  proceeded  with  the  discussion,  using  standard 
focus  group  procedures  (Krueger,  1988;  Merton,  1946,  1990) .  The 
approach  was  that  of  "retrospective  introspection"  (Merton, 

1990) ,  in  which  (in  the  present  case)  participants  were  asked, 
for  each  topic,  (a)  what  was  in  their  mind  when  they  answered  the 
questions  on  the  topic,  (b)  whether  in  responding  to  these  items 
they  recalled  being  concerned  about  confidentiality,  and,  if  so, 
(c)  what  they  were  concerned  about.  Items  in  the  survey  varied 
in  sensitivity  and  addressed  such  topics  as  satisfaction  with 
various  aspects  of  Army  life,  rating  of  supervisor's  leadership, 
rating  of  unit,  gambling,  use  of  alcohol,  and  use  of  drugs. 

Items  were  also  included  that  asked  about  the  respondent's 
military  and  social  demographics,  and  one  item  asked  the 
respondent's  SSN. 

Data  were  collected  in  May  1993.  Prior  to  the  Investigator's 
arrival  at  the  post,  each  participant  had  been  selected3  and 
assigned  to  one  of  10  small  groups  (Table  2) ;  and  there  were  two 
groups  for  each  of  five  grade/rank  categories:  E2-E4,  E5-E6,  E8, 
01-02,  and  03.  Two  forms  of  the  survey  instrument  were 
constructed:  one  with  the  demographic  items  at  the  beginning  and 


3The  Investigator  learned  later  that  any  "bad  apples"  had  been 
removed  from  the  list  of  potential  participants  before  selections 
were  made. 
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Table  1 

Privacy /Confidentiality  Questionnaire 


Markina  Instructions:  CIRCLE  THE  LETTER  IN  PRONT 

OF  THE  ANSWER  YOU  CHOOSE 

1.  How  many  Army  surveys  (including  the  survey  today)  have  you 
completed  in  the  past  12  months — that  is,  since  May  of  last  year/ 

a.  1  (The  survey  today  was  the  only  one.) 

b.  2 

c.  3 

d.  4 

e.  5 

f .  6 

g.  7 

h.  8 

i.  9 

j .  10  or  more 

2.  When  an  Army  survey  tells  you  your  answers  will  be  kept 
confidential,  do  you  assume  they  will  be  kept  confidential? 

a.  Yes,  usually 

b.  Sometimes  yes,  sometimes  no 

c.  No,  usually  not 


3.  What  about  this  particular  survey?  The  booklet  says  this 
survey  is  confidential  and  that  no  effort  will  be  made  to  trace 
your  answers  back  to  you.  Do  you  believe  this? 

a.  Yes 

b.  No 
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Table  2 


Number  and  Grade (s)  of  Participants  in  Each  Group 


Grade  Level  Number  of 
Group  of  Group  Soldiers  in 

Number  Members  This  Group 

1  . E2-E4 . 6 

2  . E2-E4 . 5 

3  . E6 . 5 

4  . E6 . 5 

5  . E8 . 1 

6  . E8 . 2 

7  . 01-02 . 5 

8  . 01-02 . 5 

9  . 03 . 3 

10 . 03 . 3 
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the  other  with  these  items  at  the  end.  For  each  of  the  five 
grade/ranJc  categories,  one  group  received  the  demographics-first 
form  and  the  other  received  the  demographics-last  form. 


RESULTS 

Responses  to  Objective  Questionnaire  Items 

Participation  in  prior  surveys.  As  indicated  above,  the 
discussion  was  preceded  by  the  administration  of  a  3-item 
questionnaire;  and  the  responses  to  these  items  are  displayed  in 
Table  3.  The  first  item  (see  Table  1)  asked  how  many  Army 
surveys  (including  the  survey  just  completed)  the  soldier  had 
completed  in  the  past  year.  The  item  was  intended  mainly  as  a 
lead-in  to  the  following  question,  but  it  also  provided 
information  on  the  respondent-burden  question  of  how  often  these 
soldiers  had  taken  part  in  an  Army  survey.  Of  the  40 
participants,  25  (63%)  had  not  during  the  preceding  year 
completed  a  survey  other  than  the  one  they  completed  for  this 
research;  and  13  (33%)  had  completed  only  one  other  survey  during 
this  period. 

Perception  of  confidentiality  in  Army  surveys.  The  second 
item  (Table  1)  asked  participants  whether  they  assumed  that 
promises  of  confidentiality  in  Army  surveys  would  be  kept.  Of 
the  40  participants,  18  (45%)  said  yes,  but  the  remaining  22 
(55%)  expressed  doubts  (either  "no"  or  "sometimes  yes  and 
sometimes  no") .  Those  most  frequently  expressing  doubts  were  the 
NCOs  (junior  and  senior) .  The  third  item  (Table  1)  asked 
participants  whether  they  believed  the  assurance  of 
confidentiality  they  had  been  given  on  the  survey  they  had  just 
completed.  Of  the  40  participants,  23  (57%)  said  yes,  but  the 
remaining  17  (42%)  said  No.  Once  again,  those  most  frequently 
expressing  doubts  were  the  NCOs. 


Participant  Comments  During  the  Discussions 

Standard  confidentiality  statement.  The  first  page  of  the 
survey  instrument  contained  a  standard  confidentiality  statement, 
and  the  Investigator  probed  group  members'  interpretation  of  this 
statement.  Some  said  they  had  read  the  statement,  and  others 
said  they  had  not.  Most,  however,  said  they  had  just  skimmed  it. 
In  fact,  most  said  they  viewed  the  inclusion  of  such  a  statement 
as  a  little  more  than  standard  boilerplate;  and,  in  varying 
degree,  most  expressed  doubt  or  uncertainty  as  to  whether  the 
assurances  would  be  honored. 

Army  letter  requesting  cooperation.  The  second  page  of  the 
survey  instrument  contained  a  letter  from  the  Deputy  Chief  of 
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Table  3 

Responses  to  Questionnaire  Items ,  Shown  Separately 
bv  Respondent  Grade/Rank _ _ _ 


QUESTION 

QUESTION 

1 

2 

3 

1 

2 

3 

RESPONDENT 

RESPONDENT 

GRADE/ RANK 

GRADE/RANK 

JUNIOR 

SENIOR 

ENLISTED 

NCOS 

1 

a 

a 

a 

22 

a 

c 

b 

2 

a 

b 

b 

23 

a 

c 

b 

3 

a 

a 

a 

24 

a 

a 

a 

4 

a 

a 

a 

5 

a 

a 

a 

6 

a 

a 

a 

LIEUTENANTS 

7 

b 

a 

a 

25 

c 

a 

a 

8 

a 

b 

a 

26 

a 

a 

a 

9 

a 

c 

b 

27 

b 

a 

a 

10 

b 

b 

b 

28 

a 

b 

a 

11 

a 

c 

b 

29 

a 

a 

a 

30 

a 

c 

b 

31 

b 

c 

b 

JUNIOR 

32 

a 

b 

a 

NCOS 

33 

a 

b 

a 

12 

b 

c 

b 

34 

b 

a 

a 

13 

a 

b 

b 

14 

a 

a 

a 

15 

a 

b 

b 

CAPTAINS 

16 

b 

a 

a 

35 

b 

a 

a 

17 

a 

c 

b 

36 

b 

a 

b 

18 

d 

c 

b 

37 

b 

c 

b 

19 

a 

b 

a 

38 

b 

a 

a 

20 

b 

b 

a 

39 

a 

a 

a 

21 

a 

b 

b 

40 

b 

c 

b 
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Staff  for  Personnel  (DCSPER)  requesting  the  respondent’s 
cooperation  in  completing  the  questionnaire  and  assuring 
confidentiality  of  participants'  responses,  and  the  Investigator 
probed  group  members's  interpretation  of  this  letter.  As  before, 
a  few  said  they  had  read  the  letter,  and  a  few  said  they  had  not; 
but  most  said  they  has  just  skimmed  it.  A  few  said  they  thought 
a  survey  coming  from  Washington  could  be  trusted  to  preserve 
confidentiality  more  than  a  survey  coming  from  a  local  commander, 
but  the  rank  of  the  letter  writer  (here  a  3-star  general)  was 
generally  viewed  as  not  making  a  difference. 

Military  and  social  demographics.  The  instrument  included  a 
variety  of  military  and  social  demographic  items.  Military 
demographics  locate  the  respondent  in  the  military  structure,  and 
the  following  items  of  this  type  were  included:  rank, 
installation,  type  of  unit,  branch  or  primary  MOS,  length  of  time 
in  the  Army,  and  SSN.  Social  demographics  locate  the  respondent 
in  the  larger  social  structure,  and  the  following  such  items  were 
included:  age,  race,  ethnicity,  gender,  and  highest  level  of 
education.  Most  participants  said  they  did  not  know  the  reason 
for  many  of  these  items  and,  as  a  result,  tended  to  be  suspicious 
of  the  purpose  these  items'  served.  A  view  frequently  voiced  was 
that  such  items  could  identify  (by  post,  by  type  of  unit,  by 
rank,  by  race,  etc.)  the  individual  who  provided  the  information 
and  that  (given  a  will  to  use  them  this  way)  the  items  could  be 
used  to  trace  responses  back  to  the  individuals  who  had  given 
them.  As  a  result,  many  respondents  said  that  if  they  were 
concerned  about  being  identified  they  would  omit  or  falsify 
responses  to  such  items  or,  alternatively,  falsify  their  response 
to  the  substantive  items  they  were  concerned  about. 

Racial/ethnic  concerns.  A  number  of  black  and  hispanic 
participants  made  an  additional  point — viz.,  that  asking  for  race 
and  ethnicity  information  in  a  survey  that  focuses  on  "bad 
things"  (discipline  problems,  use  of  drugs,  etc.)  is  seen  by  many 
as  an  effort  to  obtain  data  that  will  discredit  minority  soldiers 
by  showing  disproportionate  involvement  in  these  problems. 

Request  for  SSN.  But  far  and  away  the  most  disliked  item  was 
the  one  that  asked  respondents  for  their  SSN.  Few  of  the 
participants  were  able  to  articulate  a  legitimate  reason  for  the 
survey's  including  the  item,  and  the  fact  that  it  was  included 
reinforced  conclusions  many  had  drawn  already  regarding  the 
"true"  purpose  of  many  or  most  of  the  demographic  items. 

Demographics  first /last.  As  noted  earlier,  the  design  of  the 
research  had  provided  for  two  systematically  different  forms  of 
the  survey  instrument,  one  with  the  demographics  at  the  beginning 
and  the  other  with  the  demographics  at  the  end — the  purpose  being 
to  see  whether  the  nature  and/or  extent  of  confidentiality 
concerns  would  be  different  in  the  two  conditions.  What  happened 
was  that  confidentiality  concerns  were  voiced  at  the  outset  of 
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every  discussion — regardless  of  the  location  of  the  other 
demographics.  And  these  concerns,  at  least  initially,  were 
associated  with  the  inclusion  of  the  request  for  SSN.  When  the 
Investigator  asked  how  soldiers  usually  dealt  with  such  concerns 
he  was  told  that  the  solution  was  simple.  If  the  SSN  came  at  the 
end  of  the  survey  (and  the  substantive  questions  were  sensitive) , 
respondents  could  simply  omit  or  falsify  their  responses  to  this 
item  (they  said  they  were  unlikely  to  go  back  and  change  their 
answers  to  the  substantive  questions.)  If  the  SSN  came  at  the 
beginning  of  the  survey  they  had  a  choice:  either  omit  or  falsify 
responses  to  the  demographic  item  or  omit  or  falsify  their 
responses  to  the  substantive  questions. 

Perception  of  consequences.  An  effort  was  made  to  find  out 
just  what  participants  thought  would  happen  to  them  if  an 
assurance  of  confidentiality  were  violated.  For  the  most  part, 
participants  found  it  difficult  to  say  what  they  thought  would 

happen.  Some  said  that  their  unit  leaders  might  harass  them,  but 

ideas  about  this  tended  to  be  vague. 

Substantive  topics  in  the  survey.  The  instrument  included 
items  on  the  following  substantive  topics: 

*  Satisfaction/dissatisfaction  with  Army  life 

*  Rating  of  supervisor's  leadership 

*  Rating  of  unit 

*  Gambling 

*  Use  of  alcohol 

*  Drugs. 

In  most  cases  the  group  members  said  that  they  personally  had  no 
confidentiality  concerns  in  responding  to  any  of  these  questions; 
but  they  agreed  that  "some  soldiers"  might  have  such  concerns, 
and  the  discussion  proceeded  on  the  agreed  presumption  that  the 
discussion  concerned  other  people.  Almost  all  agreed  that  the 
rder  in  which  the  topics  were  presented  in  the  instrument  (see 
above)  was  the  order  of  their  relative  sensitivity:  satisfaction 
with  various  aspects  of  Army  life  was  the  least  sensitive,  and 
drugs  was  the  most  sensitive.  And  all  agreed  that  to  the  extent 
individuals  found  an  item  sensitive  they  would  either  omit  or 
falsify  their  response  to  that  item  or  omit  or  falsify  their 
response  to  one  or  more  demographics. 
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DISCUSSION 


The  objective  of  this  research  was  to  obtain  information  on 
whether  soldiers  believe  the  assurance  of  confidentiality  they 
receive  in  a  standard  Army  survey  and,  if  not,  what  reasons  they 
give  for  not  believing  it.  With  respect  to  the  first  part  of 
this  objective,  responses  to  the  questionnaire  items  indicated 
that  only  18  of  the  40  participants  (45%)  routinely  believe  the 
confidentiality  assurances  they  are  given  in  Army  surveys,  and 
only  23  of  them  (57%)  said  they  believed  the  assurances  they  were 
given  in  the  present  survey.  This  57%  figure  is  similar  to  the 
figure  (58%) 4  obtained  in  a  recent  Armywide  survey.  Moreover, 
the  doubt  and  uncertainty  that  participants  expressed  in  these 
questions  was  consistent  with  what  was  said  in  the  focus  group 
discussions  that  followed;  and  it  is  consistent  also  with  what 
Singer  and  Miller  (1993)  found  in  their  review  of  research 
conducted  in  civilian  populations. 

With  respect  to  the  second  part  of  the  objective,  there 
appeared  to  be  three  interrelated  reasons  for  not  (or  at  least 
not  always)  believing  the  assurances  of  confidentiality  or 
anonymity  they  received.  First,  the  soldiers  tended  to  believe 
(quite  apart  from  the  sincerity  they  were  willing  to  attribute  to 
those  conducting  the  survey)  that  it  is  simply  not  possible  to 
guarantee  complete  confidentiality  for  survey  responses.  They 
tended  to  believe  that  survey  booklets  (or  data  recorded  from 
these  booklets)  always  stand  a  chance  of  being  seen 
(inadvertently  or  otherwise)  by  individuals  whom  the  respondents 
would  not  want  to  see  them.  To  the  extent  therefore  that 
respondents  are  concerned  about  such  a  possibility  (item 
sensitivity  would  presumably  be  a  factor  here) ,  they  are  likely 
to  become  concerned  not  just  about  confidentiality  but  about 
anonymity  as  well. 

Second,  the  soldiers  tended  to  believe  that  the  nature  of  the 
background  information  requested  in  many  Army  surveys  is  such 
that  the  various  pieces  of  such  information  could  be  combined  (in 
the  event  some  one  wanted  to  do  this)  to  identify  specific 
individuals  and  connect  them  with  the  responses  they  gave  to  the 
substantive  items.  In  other  words,  in  the  mind  of  some  soldiers 
the  fact  that  they  are  not  personally  identified  in  the  survey 
booklet  does  not  mean  that  they  are  not  identifiable.  And 
depending  on  the  sensitivity  of  these  substantive  items,  such 
beliefs  by  respondents  could  have  measurable  effects  on  data 
quality. 


*Data  from  the  spring-1990  Sample  Survey  of  Military  Personnel 
(SSMP) ,  which  is  administered  by  the  Army  Personnel  Survey  Office. 
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Third,  the  soldiers  had  difficulty  articulating  a  reason  for 
the  Army's  including  the  background  items;  and  in  this  situation 
many  assumed  the  worst.  Many  of  them  assumed  such  items  had  been 
included  because  some  one  (it  wasn't  necessary  to  be  able  to  say 
who)  wanted  to  be  able  to  identify  them  individually. 

What  is  suggested  by  these  observations  is  that  for  some 
soldiers  (and,  in  the  case  of  particularly  sensitive  topics, 
possibly  most  soldiers)  concerns  about  confidentiality  lead 
easily  to  concerns  about  anonymity.  The  more  the  sensitiveness 
of  some  personally  relevant  item  stimulates  concern  about 
confidentiality,  the  more  respondents  indicated  they  would  be 
concerned  about  anonymity.  And  the  more  respondents  are 
convinced  their  responses  cannot  be  linked  to  them  personally, 
the  more  likely  they  will  be  (when  asked  a  sensitive  question)  to 
respond  candidly. 


Participant  Suggestions 

Participants  were  asked  what  they  suggested  as  a  way  of 
reducing  soldiers'  confidentiality  concerns  in  responding  to  a 
survey  like  the  one  they  had  just  completed.  Their  suggestions 
are  listed  below: 

a.  Don't  ask  for  SSN  or  name.5 

b.  Don't  ask  for  any  demographic/ background 
information. 

c.  Make  optional  the  providing/omitting  of  any 
demographic  information  requested 

d.  Don't  ask  for  any  demographic/background  information 
that  is  not  essential  for  the  stated  purpose  of  the 
survey,6 

e.  Explain  the  reason  for  items  that  are  included. 

f.  Tell  who  will  use  the  information  and  how. 


5wilson  and  Rosen  (1975) ,  based  on  the  results  of  their 
experiment  with  soldiers,  recommended  that  "the  respondent's  social 
security  number  should  not  be  required"  (p.vi). 

Wilson  and  Rosen  (1975)  also  said  that  "the  background 
information  gathered  in  anonymous  surveys  should  be  greatly  reduced 
or  eliminated  in  order  to  ensure  that  respondents  will  feel  that 
they  in  fact  will  be  anonymous."  (p.  vi.) 
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Two  participants  recalled  instances  in  which  their  belief  in 
survey  confidentiality  had  been  reduced  by  factors  that  had 
nothing  to  do  with  the  survey  instrument.  One  participant  said 
he  had  seen  a  completed  survey  form  lying  face  up  on  an  NCO's 
desk;  the  other  said  his  own  POC  had  given  him  a  survey  form 
without  an  accompanying  envelope  and  that,  as  instructed,  he  had 
returned  the  completed  form  to  the  POC  without  such  an  envelope. 


RECOMMENDATIONS  FOR  RESEARCH 

The  information  obtained  suggest  a  number  of  possibilities  for 
research.  The  following  are  variables  that  could  be  manipulated 
(alone  or  in  combination)  in  experimentally  created  survey  forms 
to  assess  their  effect  on  data  quality. 

a.  Nature  of  (explicit  or  implicit)  identifying 
information  requested7 

*  SSN 

*  Military  demographics 

*  Societal  demographics 

*  None 

b.  Option/no  option  to  omit  background  information 

c.  Large/small  number  of  demographic  items  included 

d.  Location  of  demographics  in  the  survey  instrument 
(beginning/end) 

e.  Alerting/not  alerting  respondent  to  the  inclusion  of 
background  items  in  the  survey  and  providing/not 
providing  a  plausible  justification 

f.  Printing/not  printing  tracking  number  on  answer 
sheet  (to  allow  for  follow-up  of  nonrespondents)8 


7Not  requesting  demographic  information  makes  it  impossible  to 
conduct  subgroup  analysis,  which  is  an  important  objective  of  most 
surveys . 

8lf  respondents  (or  potential  respondents)  see  a  tracking 
number  on  their  questionnaire  and  believe  it  is  intended  for 
purposes  other  than  that  of  simply  determining  who  has  and  has  not 
returned  the  questionnaire,  the  mere  inclusion  of  such  a  number 
could  undermine  other  efforts  to  assure  confidentiality  and/or 
anonymity. 
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